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(54) Data compression method for packet transmission 

(57) A method provides for improved compression 
techniques over inherently unreliable underlying net- 
works. A fixed compression dictionary is sent from a first 
terminal to a second terminal during the start up of a 
communications session between the terminals. The 
first terminal then compresses data in accordance with 
the transmitted fixed dictionary. The receiving terminal 
expands the compressed data in accordance with the 
received fixed dictionary. If packets are lost over the net- 
work such loss does not have a deleterious effect on the 
compression algorithm as in known adaptive compres- 
sion techniques. 




USER 



CD 

oo 

CO 
CO 



JNSDOCID: <EP 0933876A1_I_> 



Printed by Xerox (UK) Business Services 
2.16.7/3.6 



1 



EP0 933 876A1 



2 



Description 

BACKGROUND OF THE INVENTION 

[0001] The present invention is directed to a method 
for improving data compression used over an unreliable 
underlying network In particular, the present invention 
is directed to a method in which a compression diction- 
ary is sent at the beginning of the communication ses- 
sion and is used by the communicating parties to 
control compression and expansion of the data trans- 
mitted over the unreliable underlying network. 
[0002] It is known that it is beneficial to compress data 
streams which are transmitted over low speed lines so 
as to enhance the data flow rate over these lines. All 
modern modems perform some data compression. It is 
also known that it is beneficial to encrypt data so as to 
render it more secure thereby preventing access by par- 
ties who are not intended to receive the data. 
[0003] The Internet is a packet switching network in 
which large amounts of data can be transmitted 
between end points in a communication where com- 
pression may have some value. FIG. 1 illustrates a typi- 
cal scenario where a party accessing the Internet is 
interested in obtaining information from or communicat- 
ing with a host. The user has access to the Internet 
through an access provider, typically a point of pres- 
ence (POP), e.g.. A, and packets of data are transmitted 
via multiple routers (B to D) to and from host E. The 
question has arisen as to how to provide both encryp- 
tion for the data and compression. Encrypted data cre- 
ates a stream of apparently random bits and hence are 
inherently not compressible. One option therefore would 
be to provide encryption between points A to E without 
compression and then provide compression from 
access point A to the user without encryption. This how- 
ever, leaves a significant portion of the link vulnerable 
since it carries unencrypted data. 
[0004] Another option that has been considered is to 
perform compression first and then encrypt the com- 
pressed data and do this from end to end. There are 
some software costs associated with this scheme. How- 
ever, it can be effective in a reliable network. Typically 
the compression algorithm used in such a configuration 
is adaptive compression whereby the compression dic- 
tionary is dynamic over the course of data transmission. 
Each succeeding packet of the data transmission has 
an impact on the dictionary. Thus, the compression of a 
later packet relies, at least in part, on the packets that 
preceded it. If a packet is not received at the expansion 
terminal, or if it is lost, it is difficult to expand the subse- 
quent packets whose dictionary is in part based on the 
lost packet. While lost packets occasionally occur in a 
reliable network, this does not pose significant problems 
for the adaptive compression scheme. 
[0005] However, at its heart the Internet is based on 
an unreliable datagram network. Packets tray be 
dropped, damaged, duplicated, reordered or even 



"invented" by the network. This can pose significant 
problems for adaptive compressions. It is more likely 
that packets will not be properly received so that the 
receiving terminal will not be able to expand packets 

5 downstream of the "lost" packet. 

[0006] One proposed solution would be to periodically 
recreate the dictionary. In essence, the system could 
select a predetermined number of packets for creating a 
given dictionary and then after that number of packets 

10 has been passed downstream a new dictionary would 
begin. In this scenario, if one of the packets within a 
given group of n packets is lost then only those packets 
of data within the group and following the lost packet are 
negatively affected. The dictionary is recreated at the 

is beginning at the next n packets. This technique will only 
be effective if the number of packets "n" is carefully 
selected based on the loss characteristics of the 
medium. The inventor has discovered that for a packet 
loss probability of 5% a burst rate of 4. that is n=4, yields 

20 an effective loss rate of 12%. This effective loss rate is 
too high and can create other transmission problems. In 
fact, a high loss rate leads to even greater retransmis- 
sion of packets causing a greater traffic jam with further 
packet loss due to packet discarding operations. Fur- 

25 thermore. in TCP packet loss is treated as a sign of con- 
gestion. In response, it backs off on its transmission 
rate. This slows down throughout considerably which 
negates the beneficial effects of compression. The 
effect of a high loss rate is further described in "Round 

30 Trip Time Estimation" P. Karn and C. Partridge, ACM 
SIGCOMM-87, August 1987 and "Congestion Avoid- 
ance and Control," V. Jacobson, ACM SIGCOMM-88, 
August 1988. With a packet loss probability of 5% it 
might be possible to employ a burst rate of 2, that is, n=2 

35 since the effective loss rate would be about 7%. For a 
packet loss probability of greater than 6%, it would be 
necessary to use a burst size of 1, that is, n=1, to be 
able to keep the effective loss rate < 10%. A burst rate 
of 1 means that the entire compression dictionary must 

40 be sent with each packet of information. This is a rather 
substantial overhead cost to pay for providing compres- 
sion especially since it is known that the majority of the 
packets are of small size (40 bytes) and inclusion of the 
entire dictionary in packets of this size would be exces- 

45 sive. 

[0007] It would be beneficial if a compression tech- 
nique could be provided for unreliable underlying net- 
works without excessive overhead costs. 

50 SUMMARY OF THE INVENTION 

[0008] The present invention provides an improved 
compression scheme which permits compression over 
a packet data network that is inherently unreliable. In 
55 particular, the present invention provides that during a 
communication session between two terminals at least 
one terminal transmits a fixed compression dictionary to 
the other terminal. Subsequent communications from 
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the first terminal to the second terminal will include data 
compressed in accordance with this fixed compression 
dictionary. A second terminal will then be able to use the 
received dictionary to expand the received data. By 
using a fixed compression dictionary there is no nega- 5 
trve effect on subsequent packets of data if one of the 
preceding packets of data has been lost due to the gen- 
eral nature of the lossy medium. Furthermore, the trans- 
mission of a fixed compression dictionary, which could 
be at the beginning of the session, avoids the need for w 
transmitting an adaptive dictionary with every packet 
which creates excessive overhead costs. 
[0009] In accordance with the present invention the 
two terminals could use the same fixed compression 
dictionary for compressing and expanding data during 15 
the communication between the two terminals. Alterna- 
tively, each terminal could specify to the other terminal 
the fixed compression dictionary that the terminal will be 
using to transmit data to that other terminal. For 
instance, terminal A could have a fixed compression 20 
dictionary that it uses for compressing data transmitted 
to terminal B while terminal B could have a separate 
fixed compression dictionary for the compressed data it 
transmits to terminal A. 

[001 0] The fixed compression cfictionary could be gen- 25 
erated at the beginning of this session based on the 
type of data to be transmitted from the compressing ter- 
minal to the expanding terminal. The dictionary could be 
changed from session to session or it could remain fixed 
over all sessions. Other modifications of the content of 30 
the fixed dictionary and the way in which the terminals 
share the dictionary can be implemented. 

BRIEF DESCRIPTION OF THE DRAWINGS 

35 

[0011] 

FIG. 1 illustrates in schematic form of a network 
arrangement in which data may be compressed 
and encrypted. 40 
FIG. 2 provides a flow diagram for operations to be 
performed by a terminal compressing data in 
accordance with the present invention. 
FIG. 3 provides a flow chart showing the steps to be 
performed by the terminal receiving compressed 45 
data in accordance with the present invention. 
FIG. 4 illustrates a simple block diagram of the ter- 
minal communication configuration in which the 
present invention may be used. 

50 

DETAILED DESCRIPTION 

[0012] FIG. 4 illustrates a simple block diagram of a 
connection of two data terminals over a lossy transmis- 
sion medium. The data terminals may exchange data of 55 
many different types, e.g. p voice data, video, text, exe- 
cutable text, etc. 

[0013] While it is not significant for purposes of this 



invention which terminal initiates a session, for ease of 
discussion it will be assumed in these examples that ter- 
minal A, 40, has initiated a communication session with 
terminal B, 45. After terminal A initiates the session it 
transmits a fixed compression dictionary to terminal B 
as part of the call set up or session start up exchange. 
The compression dictionary would define to terminal B 
the dictionary that terminal A will be using to compress 
data to be transmitted to terminal B. In an alternative 
arrangement terminal A sends a pointer B and the 
pointer identifies from where terminal B can retrieve the 
compression dictionary. Upon receipt of this compres- 
sion dictionary, terminal B will store the fixed compres- 
sion dictionary for use with its communication with 
terminal A. Then, in accordance with the steps shown in 
FIG. 2, terminal A having determined the dictionary 
(step 20) and transmitted the dictionary during initializa- 
tion (step 21), compresses the data in accordance with 
the dictionary (step 22) and transmits it to terminal B. 
[0014] In accordance with the steps shown in FIG. 3, 
terminal B receives the compressed data (step 31) and 
subsequently expands the compressed data using the 
dictionary (step 32). 

[0015] In one possible implementation, terminal A 
defines the compression dictionary for all communica- 
tions between terminal A and B. that is both terminals 
compress data in accordance with the dictionary and 
both terminals expand received data in accordance with 
the dictionary. Alternatively, terminal B, during the start 
up exchange, could send a second fixed dictionary to 
terminal A thereby defining to terminal A the dictionary 
that terminal B will be using to compress data as it 
transmits information to terminal A. 
[001 6] The compression dictionary that is transmitted 
could be generated in a number of different ways. First, 
the terminal transmitting the compression dictionary 
may simply access a portion of its memory which stores 
a dictionary which remains static over time and is used 
for all communications by that terminal. Alternatively, 
the terminal, for example, terminal A, could store multi- 
ple static dictionaries: each dictionary would be associ- 
ated with a particular type of data transmission that the 
terminal will be engaging in. Then, at the beginning of 
the session, the terminal could select the appropriate 
static dictionary based on the data type to be employed 
in that given transmission. In another modification the 
terminal transmitting the dictionary could generate a 
new dictionary at the beginning of every session. The 
terminal could use software previously loaded on the 
terminal or it could use software downloaded from some 
other terminal in the packet switching network to create 
a dictionary that is particularly suited to this session. 
[0017] The dictionary can actually be constituted by 
different portions, which themselves may be either static 
or generated on a per session basis. Examples of pos- 
sible components of the dictionary would be: a starting 
dictionary that could be based on the type of text being 
transmitted (language text or executable text); informa- 
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tion regarding the data type involved in the transmission 
between the two terminals; TCP connection state infor- 
mation, for example, an initial sequence number to be 
used, which can be used as part of TCP header com- 
pression techniques which are already known. Exam- 
ples of the compression techniques are described in a 
paper entitled "Compressing JCP/IP Headers for Low- 
Speed Serial Links," by V. Jacobson, Feb. 1990, hereby 
incorporated herein by reference. 
[0018] The transmission of the fixed compression dic- 
tionary at the start of the session facilitates compres- 
sion over the lossy medium without exorbitant overhead 
costs. 

[001 9] The above description has referred to a simple 
two terminal connection. It should be noted that the 
present invention is also applicable to a multi-terminal 
connection, for example a multi-cast operation. The 
implementation would simply be adjusted to advise the 
multiple terminals of the fixed dictionary or dictionaries 
to be used. This could be done at the terminal point 
where each terminal joins the multi-cast. 
[0020] The present invention is not limited to the arena 
of the Internet. Instead, it could be employed over any 
inherently unreliable underlying network over which 
packets of data are being transmitted. 
[0021] In addition, it might be possible to use the fixed 
compression dictionary transmitted at the beginning of 
the session as a primary or base line dictionary which 
could be supplemented by adaptive compression 
schemes. As a consequence, as packets are created 
and transmitted the dctionary could be expanded. But if 
packets are lost during the course of the transmission 
the basic dictionary will still be available for expansion of 
the received packets that follow the lost packet. 
[0022] In accordance with the present invention the 
flow of data between two or more terminals can be com- 
pressed and expanded even though they are impressed 
upon inherently unreliable underlying networks. 
[0023] Where technical features mentioned in any 
claim are followed by reference signs, those reference 
signs have been included for the sole purpose of 
increasing the intelligibility of the claims and accord- 
ingly, such reference signs do not have any limiting 
effect on the scope of each element identified by way of 
example by such reference signs. 

Claims 
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tionary; and 

transmitting the compressed packet data from 
the first terminal to the second terminal over 
the packet data transmission medium. 

2. The method of claim 1 wherein said step of trans- 
mitting the fixed compression dictionary comprises 
the substeps of: 

transmitting a pointer from the first terminal to 
the second terminal; and 
retrieving to the second terminal the fixed com- 
pression dictionary using said pointer. 

3. The method of claim 1 wherein said step of trans- 
mitting the fixed compression dictionary comprises 
the substep of transmitting the fixed compression 
dictionary from the first terminal to the second ter- 
minal. 

4. The method of claim 1 wherein said fixed compres- 
sion dictionary includes a starting data compres- 
sion dictionary. 

The method of claim 4 wherein said fixed compres- 
sion dictionary further includes a header compres- 
sion dictionary. 

A method for receiving at a first terminal, a data 
transmission from a second terminal, over a packet- 
data transmission medium, the method comprising 
the steps of: 

receiving a fixed compression dictionary from 
the first terminal; 

receiving from the first terminal packet-data 
compressed in accordance with said fixed 
compression dictionary; and 
decompressing the received packet data in 
accordance with the received fixed compres- 
sion dictionary. 

The method of claim 6 wherein said f ixed compres- 
sion dictionary includes a starting data compres- 
sion dictionary. 



25 5. 



30 



8. 



1 . A method for data transmission between two termi- 
nals over a packet-data transmission medium, the so 
method comprising the steps of: 9 * 



initiating a communication session over the 
packet data transmission medium; 
transmitting a fixed compression dictionary 
after prompting from a first terminal; 
at said first terminal, compressing packet data 
in accordance with said f ixed compression dic- 



55 



The method of claim 7 wherein said fixed compres- 
sion dictionary further includes a header compres- 
sion dictionary. 

A method for communication of data between a first 
terminal and a second terminal over a packet data 
transmission medium, the method comprising the 
steps of: 

initiating a communication session between the 
first and second terminals over the packet data 
transmission medium; 
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transmitting a fixed compression dictionary 
from the first terminal to the second terminal; 
receiving from the first terminal packet-data 
compressed in accordance with said first com- 
pression dictionary; and 
decompressing, at the second terminal, the 
received compressed packet data in accord- 
ance with said fixed compression dictionary. 

10. The method of claim 9 comprising the further step 
of: 



10 



compressing packet data at said first terminal 
in accordance with said fixed compression dic- 
tionary; and rs 
transmitting the packet-data compressed at 
said first terminal to said second terminal. 



accordance with the type of data to be transmitted. 

18. The method of claim 15 further comprises the step 
of said first terminal compressing data in accord- 
ance with said fixed compression dictionary for 
transmission to said second terminal. 

19. The method of claim 15 further comprising the 
steps of: 

transmitting a second fixed compression dic- 
tionary from the first terminal to the second ter- 
minal; and 

compressing data in accordance with said sec- 
ond fixed compression dictionary for transmit- 
ting to said second terminal. 



1 1 . The method of claim 9 wherein said fixed compres- 
sion dictionary includes a starting data compres- 
sion dictionary. 



20 



12. The method of claim 11 wherein said fixed com- 
pression dictionary further includes a header com- 
pression dictionary. 25 



13. The method of claim 10 wherein said fixed com- 
pression dictionary includes a starting data com- 
pression dictionary. 

14. The method of claim 13 wherein said fixed com- 
pression dictionary her includes a header compres- 
sion dictionary. 

15. In a data packet transmission system including a 
first and a second terminal, a communication 
method comprising the steps of: 

initializing a communication session between 
the first and second terminals, wherein during 
the initializing the first terminal receives a fixed 
compressing dictionary from the second termi- 
nal; 

receiving packet-data from the first terminal, 
the received packet-data being compressed in 
accordance with said fixed compression dic- 
tionary; and 

decompressing the received packet data in 
accordance with the received fixed compres- 
sion dictionary. 
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16. The method of claim 15 wherein the content of said 
fixed compression dictionary is selected by the sec- 
ond terminal in accordance with the type of data to 

be transmitted. 55 

1 7. The method of claim 1 5 wherein the content of said 
fixed compression dictionary is generated in 
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